Relative perturbation theory: IV. sin 2θ theorems

AbstractThe double angle theorems of Davis and Kahan bound the change in an invariant subspace when a Hermitian matrix A is subject to an additive perturbation A→Ã=A+ΔA. This paper supplies analogous results when A is subject to a congruential, or multiplicative, perturbation A→Ã=D*AD. The relative gaps that appear in the bounds involve the spectrum of only one matrix, either A or Ã, in contrast to the gaps that appear in the single angle bounds.The double angle theorems do not directly bound the difference between the old invariant subspace S and the new one S̃ but instead bound the difference between S̃ and its reflection JS̃ where the mirror is S and J reverses S⊥, the orthogonal complement of S. The double angle bounds are proportional to the departure from the identity and from orthogonality of the matrix D̃=defD−1JDJ. Note that D̃ is invariant under the transformation D→D/αforα≠0, whereas the single angle theorems give bounds proportional to D's departure from the identity and from orthogonality.The corresponding results for the singular value problem when a (nonsquare) matrix B is perturbed to B̃=D*1BD2 are also presented

Locally Unitarily Invariantizable NEPv and Convergence Analysis of SCF

We consider a class of eigenvector-dependent nonlinear eigenvalue problems (NEPv) without the unitary invariance property. Those NEPv commonly arise as the first-order optimality conditions of a particular type of optimization problems over the Stiefel manifold, and previously, special cases have been studied in the literature. Two necessary conditions, a definiteness condition and a rank-preserving condition, on an eigenbasis matrix of the NEPv that is a global optimizer of the associated optimization problem are revealed, where the definiteness condition has been known for the special cases previously investigated. We show that, locally close to the eigenbasis matrix satisfying both necessary conditions, the NEPv can be reformulated as a unitarily invariant NEPv, the so-called aligned NEPv, through a basis alignment operation -- in other words, the NEPv is locally unitarily invariantizable. Numerically, the NEPv is naturally solved by an SCF-type iteration. By exploiting the differentiability of the coefficient matrix of the aligned NEPv, we establish a closed-form local convergence rate for the SCF-type iteration and analyze its level-shifted variant. Numerical experiments confirm our theoretical results.Comment: 38 pages, 11 figure

Nearly Optimal Stochastic Approximation for Online Principal Subspace Estimation

Processing streaming data as they arrive is often necessary for high dimensional data analysis. In this paper, we analyse the convergence of a subspace online PCA iteration, as a followup of the recent work of Li, Wang, Liu, and Zhang [Math. Program., Ser. B, DOI 10.1007/s10107-017-1182-z] who considered the case for the most significant principal component only, i.e., a single vector. Under the sub-Gaussian assumption, we obtain a finite-sample error bound that closely matches the minimax information lower bound of Vu and Lei [Ann. Statist. 41:6 (2013), 2905-2947].Comment: 37 page